feat(models): add StreamPETR with production three-stage training - #115
Closed
vividf wants to merge 8 commits into
Closed
feat(models): add StreamPETR with production three-stage training#115vividf wants to merge 8 commits into
vividf wants to merge 8 commits into
Conversation
GroupStreamingSampler walks whole scenes frame by frame per dataloader lane, the shared multiview datamodule wires it into every split, and ResizeCropFlipRotImage folds resize/crop/flip into the camera intrinsics. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
VoVNet-99 multiscale backbone, grid-mask augmentation, the NMS-free 3D bbox coder, box L1 match cost, and the temporal-query helper library (positional embeddings, memory refresh, top-k gather, ego-frame point transforms). Signed-off-by: vividf <yihsiang.fang@tier4.jp>
The query-based head with temporal memory (pre/post memory update, top-k propagation, denoising queries, geometry-aware position embeddings) and the model wrapper with the three-module ONNX export ABI. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
Signed-off-by: vividf <yihsiang.fang@tier4.jp>
Partial-annotation status plumbed through the multiview datasets, scene contiguity validated at load time, shared-tensor-aware checkpoint loading, epoch-end validation for variable-length streaming samplers, and the iteration-warmup epoch-cosine LR schedule. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
…ego poses LoadAnnotations2DFromBoxes3D projects the augmented 3D boxes onto every camera for the auxiliary 2D head; the camera geometric augmentations fold their transforms into the ego poses so the temporal memory warp stays consistent; train-time camera-order shuffling follows the reference recipe. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
… normalization Focal-PETR-style FocalHead2D supervising the image features during training, CPFPN as the reference neck, traffic_cone/barrier partial-ignore as classification-column weights shared by both heads, and cross-rank positive-count normalization (mmdetection reduce_mean) with real multi-process regression tests. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
Native three-stage flow (nuScenes pretrain -> T4 base -> j6gen2) with each stage pinned to its accepted recipe, the 2.8.1 info generation (7 classes with per-frame annotation status), a shared FocalHead2D config fragment, and the reworked model documentation. Signed-off-by: vividf <yihsiang.fang@tier4.jp>
vividf
marked this pull request as draft
September 3, 2026 17:00
Collaborator
Author
|
Superseded: landing this incrementally instead — one layer per PR, starting with the scene-streaming data pipeline. Each next layer will be opened against main once the previous one merges, so every PR stays a small, self-contained diff. The full stack is reviewable per-layer on the fork (vividf#7–#10, #3–#6). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds StreamPETR — a camera-only, temporally-modeled 3D object detector — end to end: the scene-streaming data pipeline its memory bank requires, the model (VoVNet backbone, CPFPN neck, query head with temporal memory, denoising queries, three-module ONNX export ABI), an auxiliary 2D FocalHead for training-time feature supervision, traffic_cone/barrier partial-ignore, DDP-correct loss normalization, and the production three-stage training configs (nuScenes pretrain → T4 base → j6gen2) pinned to the accepted runs.
Accepted results: stage 1 nuScenes val mAP 0.5031, stage 2 T4-base val mAP 0.4420, stage 3 j6gen2 test mAP 0.5025 (0–121 m bucket, comparable to AWML T4MetricV2 — see the evaluation-alignment notes in
docs/models/streampetr.md).How to review
Direct branch pushes to this repository are currently restricted (sec-inc-85 ruleset), so this lands as one PR instead of a stacked series. The history is already split into 8 self-contained commits, and the same 8 layers exist as a reviewable PR stack on the fork — each fork PR shows only its own layer's diff, with a detailed description, and was tested standalone:
Reviewing commit-by-commit on this PR gives the same per-layer diffs.
Testing